Camera module having multi-cell structure and portable communication device including the same

ABSTRACT

A portable communication device configured to activate an image capturing function of a camera module. When the image capturing function is activated, use an image sensor to obtain first raw image data including a specified channel pattern, generated by binning image data obtained from the specified number of image pixels corresponding to the same channel. Display a first image generated based on the first raw image data on a display. While the image capturing function is activated, receive an input associated with adjustment of a zoom setting. Use the image sensor to obtain second raw image data having the specified channel pattern, generated by re-mosaicing image data obtained from some of the plurality of image pixels corresponding to a portion of the image sensor based on the input. Display a second image generated based on the second raw image data on the display.

CROSS-REFERENCE TO RELATED APPLICATION

This application is based on and claims priority under 35 U.S.C. § 119to Korean Patent Application No. 10-2018-0167437 filed on Dec. 21, 2018,in the Korean Intellectual Property Office, the disclosure of which isincorporated by reference herein its entirety.

BACKGROUND 1. Field

The disclosure relates to a zoom function providing technology using animage sensor.

2. Description of Related Art

A camera may include a multifocal lens that is able to adjust a distancebetween lenses included in a body tube and a unifocal lens that is notable to adjust the distance between the lenses included in the bodytube. A portable electronic device adopts a camera (hereinafter,referred to as a unifocal camera) including the unifocal lens due to aspace constraint.

The portable electronic device may include a single unifocal camera anduse a digital zoom function that virtually generates some of imagepixels to provide a zoom effect. Alternatively, the portable electronicdevice may include a plurality of unifocal cameras having differentfocal distances, and switch a camera providing an image from oneunifocal camera to another unifocal camera to provide a zoom effectsimilar to an optical zoom function.

The above information is presented as background information only toassist with an understanding of the disclosure. No determination hasbeen made, and no assertion is made, as to whether any of the abovemight be applicable as prior art with regard to the disclosure.

SUMMARY

When an electronic device provides a digital zoom function using asingle unifocal camera, an image quality of the electronic device may belowered after adjustment of zoom setting. In addition, when theelectronic device provides a zoom function using a plurality of unifocalcameras, not only a cost of the electronic device and complexity may beincreased, but also only a zoom function for a center region of an imagesensor may be provided.

Various embodiments disclosed in the disclosure provide a camera modulehaving a multi-cell structure and a portable communication deviceincluding the camera module that may lower image resolution degradationwhen adjusting the zoom setting.

Aspects of the disclosure are to address at least the above-mentionedproblems and/or disadvantages and to provide at least the advantagesdescribed below. Accordingly, an aspect of the disclosure is to providea portable communication device including a camera module including animage sensor including a plurality of image pixels and a control circuitfor controlling the image sensor, a display, and a processor, whereinthe processor activates an image capturing function of the cameramodule, when the image capturing function is activated, uses the imagesensor to obtain first raw image data having a specified channel patterngenerated by binning image data obtained from the specified number ofimage pixels corresponding to the same channel with respect to theplurality of image pixels, displays a first image generated based on thefirst raw image data on the display, while the image capturing functionis activated, receives an input associated with adjustment of zoomsetting, uses the image sensor to obtain second raw image data havingthe specified channel pattern generated by re-mosaicing image dataobtained from some of the plurality of image pixels corresponding to aportion of the image sensor based at least on the input, and displays asecond image generated based on the second raw image data on thedisplay.

Another aspect of the disclosure is to provide a portable communicationdevice including a camera module including an image sensor including aplurality of image pixels and a control circuit for controlling theimage sensor, a display, and a processor, wherein the processor receivesan input associated with adjustment of zoom setting related tocapturing, uses the image sensor to obtain first raw image datagenerated by binning image data obtained from the first set number ofpixels corresponding to the same channel with respect to the pluralityof image pixels based on the zoom setting being within a specified firstmagnification range, and displays a first image generated using theobtained first raw image data on the display, and uses the image sensorto obtain second raw image data generated by re-mosaicing image dataobtained from some of the plurality of image pixels corresponding to aportion of the image sensor based on the zoom setting being within aspecified second magnification range, and displays a second imagegenerated using the obtained second raw image data on the display.

Another aspect of the disclosure is to provide a camera module includingan image sensor including a plurality of image pixels, and a controlcircuit for controlling the image sensor, wherein the control circuitactivates an image capturing function of the image sensor, when theimage capturing function is activated, uses the image sensor to generatefirst raw image data having a specified channel pattern generated bybinning image data obtained from the specified number of image pixelscorresponding to the same channel with respect to the plurality of imagepixels, transmits the generated first raw image data to an externalprocessor electrically connected to the control circuit, receives arequest associated with zoom setting while the image capturing functionis activated, uses the image sensor to generate second raw image datahaving the specified channel pattern by re-mosaicing image data for someof the plurality of image pixels corresponding to a portion of the imagesensor based at least on the request, and transmits the generated secondraw image data to the external processor.

Other aspects, advantages, and salient features of the disclosure willbecome apparent to those skilled in the art from the following detaileddescription, which, taken in conjunction with the annexed drawings,discloses various embodiments of the disclosure.

Before undertaking the DETAILED DESCRIPTION below, it may beadvantageous to set forth definitions of certain words and phrases usedthroughout this patent document: the terms “include” and “comprise,” aswell as derivatives thereof, mean inclusion without limitation; the term“or,” is inclusive, meaning and/or; the phrases “associated with” and“associated therewith,” as well as derivatives thereof, may mean toinclude, be included within, interconnect with, contain, be containedwithin, connect to or with, couple to or with, be communicable with,cooperate with, interleave, juxtapose, be proximate to, be bound to orwith, have, have a property of, or the like; and the term “controller”means any device, system or part thereof that controls at least oneoperation, such a device may be implemented in hardware, firmware orsoftware, or some combination of at least two of the same. It should benoted that the functionality associated with any particular controllermay be centralized or distributed, whether locally or remotely.

Moreover, various functions described below can be implemented orsupported by one or more computer programs, each of which is formed fromcomputer readable program code and embodied in a computer readablemedium. The terms “application” and “program” refer to one or morecomputer programs, software components, sets of instructions,procedures, functions, objects, classes, instances, related data, or aportion thereof adapted for implementation in a suitable computerreadable program code. The phrase “computer readable program code”includes any type of computer code, including source code, object code,and executable code. The phrase “computer readable medium” includes anytype of medium capable of being accessed by a computer, such as readonly memory (ROM), random access memory (RAM), a hard disk drive, acompact disc (CD), a digital video disc (DVD), or any other type ofmemory. A “non-transitory” computer readable medium excludes wired,wireless, optical, or other communication links that transporttransitory electrical or other signals. A non-transitory computerreadable medium includes media where data can be permanently stored andmedia where data can be stored and later overwritten, such as arewritable optical disc or an erasable memory device.

Definitions for certain words and phrases are provided throughout thispatent document, those of ordinary skill in the art should understandthat in many, if not most instances, such definitions apply to prior, aswell as future uses of such defined words and phrases.

BRIEF DESCRIPTION OF THE DRAWINGS

For a more complete understanding of the present disclosure and itsadvantages, reference is now made to the following description taken inconjunction with the accompanying drawings, in which like referencenumerals represent like parts:

FIG. 1 illustrates a portable communication device capable of adjustinga zoom magnification according to an embodiment;

FIG. 2 illustrates a block diagram of a portable communication deviceaccording to an embodiment;

FIG. 3A illustrates a configuration diagram of a camera module accordingto an embodiment;

FIG. 3B illustrates image pixels included in a single channel accordingto an embodiment;

FIG. 4 illustrates a process of generating first raw image dataaccording to an embodiment;

FIG. 5 illustrates a process of generating second raw image dataaccording to an embodiment;

FIG. 6 illustrates an example of a method for controlling an imagesensor by a processor according to an embodiment;

FIG. 7 illustrates an example of a method for controlling an imagesensor by a processor according to an embodiment;

FIG. 8 illustrates a method for controlling an image sensor by a controlcircuit according to an embodiment;

FIG. 9 illustrates a block diagram of an electronic device in a networkenvironment according to various embodiments; and

FIG. 10 illustrates a block diagram 1000 of the camera module accordingto various embodiments.

In the description of the drawings, the same or similar referencenumerals may be used for the same or similar components.

DETAILED DESCRIPTION

FIGS. 1 through 10, discussed below, and the various embodiments used todescribe the principles of the present disclosure in this patentdocument are by way of illustration only and should not be construed inany way to limit the scope of the disclosure. Those skilled in the artwill understand that the principles of the present disclosure may beimplemented in any suitably arranged system or device.

FIG. 1 illustrates a portable communication device capable of adjustinga zoom magnification according to an embodiment.

Referring to FIG. 1, in operation 101, in a preview mode, a portablecommunication device 100 may output, on a display 140, a preview imagegenerated based on data obtained in an entire region of an image sensorincluded in a camera module 110. The camera module 110 may be exposedthrough a rear surface of the portable communication device 100, forexample. For example, the portable communication device 100 may obtaindata from the image sensor, bin at least a portion of the obtained datato generate first raw image data, and generate first image based on thefirst raw image data and output the generated first image on the display140.

When an input associated with adjustment of zoom setting is received inoperation 101, the portable communication device 100 may output, on thedisplay 140, a preview image generated based on data obtained in theportion of the image sensor in operation 102. The input associated withthe adjustment of the zoom setting may include, for example, a pinch-outinput in which a space between two fingers, which touched the display140 (e.g., touch screen display), is widened. For example, the portablecommunication device 100 may obtain the data from the portion of theimage sensor, re-mosaic at least a portion of the obtained data togenerate second raw image data, and generate second image based on thesecond raw image data and output the generated second image on thedisplay 140. Additionally, or alternatively, the portable communicationdevice 100 may store the generated image. For example, the portablecommunication device 100 may store the generated image in response to aninput associated with image capturing.

FIG. 2 illustrates a block diagram of a portable communication deviceaccording to an embodiment.

Referring to FIG. 2, according to one embodiment, the portablecommunication device 100 may include the camera module 110, the display140, and a processor 150. In one embodiment, the portable communicationdevice 100 may omit some components or further include additionalcomponents. For example, the portable communication device 100 mayfurther include a memory 130. In one embodiment, some of components ofthe portable communication device 100 may be combined with each other asa single entity, but the single entity may perform the same functions ofthe corresponding components before the combination.

According to an embodiment, the camera module 110 may include an imagesensor 113 including a plurality of image pixels and a control circuit115 for controlling the image sensor 113. The image sensor 113 mayinclude the plurality of image pixels formed in a multi-cell structurein which data for the specified number of image pixels corresponding tothe same channel may be binned. For example, the plurality of imagepixels may be included in a single channel on the N×N pixels basis. Tothis end, a plurality of ((M×M)/(N×N)) color filters may be arranged onthe plurality of image pixels, and one color filter may be disposed onthe N×N pixels. A case in which the N×N pixels are included in thesingle channel will be described as an example. The N×N pixelscorresponding to the same channel may share an output node (e.g., afloating diffusion area). In the disclosure, a case in which theplurality of image pixels is included in the single channel on the 2×2pixels basis will be described as an example. However, the disclosure isnot limited thereto. For example, the plurality of image pixels may beincluded in the single channel on a matrix form of the 3×3, 4×4, orgreater basis. The control circuit 115 may use the image sensor 113 toobtain image data from at least some of the plurality of image pixels,and process (e.g., bin, re-mosaic) the obtained image data to generateraw image data. For example, the control circuit 115 may adjust a timingof obtaining the image data from the plurality of image pixels to bin ornot bin the image data.

According to one embodiment, the display 140 may display variouscontents (e.g., a text, an image, a video, an icon, and/or a symbol, orthe like), for example. The display 140 may include, for example, aliquid crystal display (LCD), a light emitting diode (LED) display, anorganic light emitting diode (OLED) display, or an electronic paperdisplay. The display 140 may be, for example, a touch screen displaycapable of detecting a user's touch.

The processor 150 may execute operations or data processing associatedwith control and/or communication of at least one of other components ofthe portable communication device 100 using instructions stored in thememory 130. The processor 150 may include at least one of a graphicprocessing unit (GPU), an application processor, or an image processor.

According to one embodiment, when an input associated with activation ofan image capturing function is received, the processor 150 may activatethe image capturing function of the camera module 110. The inputassociated with the activation of the image capturing function may be,for example, a user's input of selecting (e.g., touching) an iconassociated with the activation of the image capturing function. Forexample, the processor 150 may transmit a request associated with theactivation of the image capturing function to the control circuit 115such that driving power is supplied to the camera module 110 and thecontrol circuit 115 activates the image capturing function of the imagesensor 113.

According to one embodiment, when the image capturing function isactivated, the processor 150 may use the image sensor 113 to obtain thefirst raw image data having a specified channel pattern, which isgenerated by binning the image data obtained from the first set number(or the specified number) of image pixels corresponding to the samechannel with respect to the plurality of image pixels. The first setnumber may be the total number of the image pixels included in thesingle channel. The specified channel pattern may be a bayer channelpattern in which R image data and G image data alternately appear on oddrow lines, and the G image data and B image data alternately appear oneven row lines. For example, the processor 150 may use the controlcircuit 115 to control the image sensor 113 such that image data for thefirst set number of the image pixels corresponding to the same channelamong the plurality of image pixels are binned and then read-out.

According to one embodiment, the processor 150 may display the firstimage generated based on the first raw image data on the display 140.For example, the processor 150 color-interpolates the first raw imagedata such that each pixel of the first raw image data includes all of Rinformation, G information, and B information, and then converts thecolor-interpolated first raw image data into a specified format (e.g., aYUV format) to generate the first image data. The processor 150 maygenerate the first image based on the generated first image data anddisplay the first image on the display 140.

According to one embodiment, the processor 150 may receive the inputassociated with the adjustment of the zoom setting while the imagecapturing function is activated. The input associated with theadjustment of the zoom setting is an input of requesting the adjustmentof the zoom magnification, which may include a zoom in and a zoom out.For example, the input associated with the adjustment of the zoomsetting may include a first input of widening the space between the twofingers, which touched the touch screen display 140. As another example,the input associated with the adjustment of the zoom setting may includea second input of selecting a first menu of setting a zoom region or asecond menu of setting a zoom magnification. In the disclosure, anexample in which the input associated with the adjustment of the zoomsetting is the first input will be described as an example.

According to one embodiment, when the input associated with theadjustment of the zoom setting is received, the processor 150 maydetermine the zoom magnification based on the received input. Forexample, the processor 150 may determine the zoom magnification based ona change in the space between the two fingers touched the touch screendisplay 140. In this regard, when the space between the two touchedfingers is widened, the processor 150 may increase the zoommagnification based on a degree of widening. Further, when the spacebetween the two touched fingers is reduced, the processor 150 maydecrease the zoom magnification based on a degree of reduction.

According to one embodiment, when the input associated with theadjustment of the zoom setting is received, the processor 150 maydetermine image pixels to be used for obtaining a zoomed image (some ofthe image pixels) among the plurality of image pixels based on thereceived input. For example, position information of the some of theimage pixels mapped on a selected position (e.g., row information andcolumn information of the some of the image pixels) may be stored in thememory 130. Further, as the processor 150 identifies the positioninformation of the some of the image pixels corresponding to theselected position (e.g., a center position between the two fingers) fromthe memory 130, the processor 150 may determine the image pixels to beused for obtaining the zoomed image. The some of the image pixels mappedon the selected position may be different for each zoom magnification.For example, as the zoom magnification is higher, the total number ofsome of the image pixels may be smaller.

According to one embodiment, when it is identified that the zoom setting(or the zoom magnification) is within a specified first magnificationrange, the processor 150 may use the image sensor 113 to obtain thefirst raw image data generated by binning the image data obtained fromthe first set number of pixels corresponding to the same channel amongthe plurality of image pixels, and display the first image generatedusing the obtained first raw image data on the display 140. The firstmagnification range may be about 1 time, for example.

According to one embodiment, when it is identified that the zoom settingis within a specified second magnification range, the processor 150 mayuse the image sensor 113 to obtain the second raw image data generatedby re-mosaicing the image data obtained from the some of the pluralityof image pixels corresponding to a portion of the image sensor 113. Thesecond magnification range may be, for example, N times corresponding tothe number of horizontal image pixels or vertical image pixels includedin the single channel. For example, when it is identified that the zoommagnification is within the second magnification range based on theinput associated with the adjustment of the zoom setting, the processor150 may identify the position information of the some of the imagepixels based on the zoom magnification, which is mapped on the selectedposition based on the input. The processor 150 may use the image sensor113 and the control circuit 115 to obtain the image data for the some ofthe image pixels, and use the control circuit 115 to re-mosaic the imagedata obtained from the some of the image pixels, thereby generating andobtaining the second raw image data having a specified channel pattern.The second raw image data may be image data having the same resolutionas the first raw image data.

According to one embodiment, the processor 150 may obtain third rawimage data instead of the second raw image data when the zoom setting iswithin the second magnification range, based on at least one of ambientilluminance or a signal to noise ratio (SNR) of the image. For example,when it is identified that the zoom setting is within the secondmagnification range, the processor 150 may identify the ambientilluminance using an illuminance sensor 120 and determine whether theambient illuminance is within a specified illuminance range. Thespecified illuminance range may be an illuminance range of, for example,equal to or above a reference illuminance (e.g., about 50 lux) ofdetermining whether the illuminance is low. When the ambient illuminanceis within the specified illuminance range, the processor 150 maygenerate the above-described second raw image data. Further, when theambient illuminance is not within the specified illuminance range, theprocessor 150 may use the image sensor 113 to obtain the third raw imagedata generated by at least binning image data obtained from thespecified number of image pixels corresponding to the same channel amongthe some of the image pixels corresponding to the portion of the imagesensor 113. As another example, when it is identified that the zoomsetting is within the second magnification range, the processor 150 mayidentify a signal to noise ratio of the first raw image data anddetermine whether the signal to noise ratio is within a specified ratiorange. The specified ratio range may be determined experimentally as acriterion for determining a deterioration of an image quality of thefirst raw image data. When the signal to noise ratio is within thespecified ratio range, the processor 150 may use the image sensor 113 toobtain the above-described second raw image data. In variousembodiments, the processor 150 may determine whether to obtain thesecond raw image data even after, based on the signal to noise ratio ofthe obtained second raw image data. When the signal to noise ratio isnot within the specified ratio range, the processor 150 may use theimage sensor 113 to obtain the third raw image data generated by binningthe data obtained from the specified number of image pixelscorresponding to the same channel among the some of the plurality ofimage pixels corresponding to the portion of the image sensor 113. Asanother example, when it is identified that the zoom setting is withinthe second magnification range, the processor 150 may determine whetherthe ambient illuminance is within the specified illuminance range andwhether the signal to noise ratio of the first image is within thespecified ratio range. When the ambient illuminance is within thespecified illuminance range and the signal to noise ratio of the firstimage is within the specified ratio range, the processor 150 may use theimage sensor 113 to obtain the above-described second raw image data.When the ambient illuminance is not within the specified illuminancerange, or when the signal to noise ratio of the first image is notwithin the specified ratio range, the processor 150 may use the imagesensor 113 to obtain the above-described third raw image data. Theprocessor 150 may display the second image generated based on the secondraw image data or the third image generated based on the third raw imagedata on the display 140. According to the embodiment described above, asthe processor 150 displays a third image (a preview image) using thethird raw image data generated by binning the image data for the some ofthe image pixels instead of the second raw image, the processor 150 maydisplay a preview image having a higher image quality based on asurrounding situation or an image quality.

According to one embodiment, the processor 150 may store the previewimage (one of the first to third images) in the memory 130. In addition,the processor 150 may store the image generated in response to the inputassociated with the image capturing in the memory 130.

According to various embodiments, when the ambient illuminance is notwithin the specified illuminance range or the signal to noise ratio ofthe image is not within the specified ratio range, the processor 150 mayoutput a screen indicating that the adjustment of the zoom magnificationis not available in response to the input associated with the adjustmentof the zoom setting and not adjust the zoom magnification, or provide adigital zoom based on the first raw image data or the second raw imagedata.

According to various embodiments, based on the zoom setting being withina specified third magnification range, the processor 150 may use theimage sensor 113 to obtain fourth raw image data in which image dataobtained from pixels of the second set number, which is different fromthe first set number, corresponding to the same channel among theplurality of image pixels is at least binned. The second set number maybe the number of some of the image pixels included in each channel. Thethird magnification range may include a magnification of greater thanabout 1 time and less than N times. For example, when the plurality ofimage pixels is included in a single channel on the 3×3 pixels basis,the second magnification range may include magnification of about 3times, and the third magnification range may include magnification ofabout 2 times. Additionally or alternatively, the processor 150 may usethe image sensor 113 to obtain the fourth raw image data, which isgenerated by binning the image data obtained from the second set numberof pixels corresponding to the same channel with respect to some of theremaining image pixels among the plurality of image pixels and furtherre-mosaicing the binned image data. The processor 150 may display fourthimage generated using the obtained fourth raw image data on the display140.

According to the above embodiment, in the portable communication device100, a resolution of the raw image data after the adjustment of the zoomsetting (e.g., the second raw image data) may not be lowered compared tothat of the raw image data before the adjustment of the zoom setting(e.g., the first raw image data) through the binning or the re-mosaicingof the image data for the plurality of image pixels of the image sensor113.

According to one embodiment, a portable communication device (e.g., theportable communication device 100 of FIG. 2) may include a camera module(the camera module 110 of FIG. 2) including an image sensor (e.g., theimage sensor 113 of FIG. 2) including a plurality of image pixels and acontrol circuit (e.g., the control circuit 115 of FIG. 2) forcontrolling the image sensor; a display (e.g., the display 140 of FIG.2); and a processor (e.g., the processor 150 of FIG. 2). The processormay activate an image capturing function of the camera module, when theimage capturing function is activated, use the image sensor to obtainfirst raw image data having a specified channel pattern, which isgenerated by binning image data obtained from the specified number ofimage pixels corresponding to the same channel with respect to theplurality of image pixels, display a first image generated based on thefirst raw image data on the display, receive an input associated withadjustment of zoom setting while the image capturing function isactivated, use the image sensor to obtain second raw image data havingthe specified channel pattern, which is generated by re-mosaicing imagedata obtained from some of the plurality of image pixels correspondingto a portion of the image sensor, based at least on the input, anddisplay a second image generated based on the second raw image data onthe display.

The processor may identify a position selected as a zoom region and zoommagnification based at least on the input associated with the adjustmentof the zoom setting, and the some of the image pixels may be imagepixels corresponding to the identified zoom magnification, mapped on theselected position.

The second raw image data may be raw image data generated to have aresolution corresponding to a resolution of the first raw image data.

The processor may identify an ambient illuminance and obtain the secondraw image data using the image sensor when the ambient illuminance iswithin a specified illuminance range.

When the ambient illuminance is not within the specified illuminancerange, the processor may use the image sensor to obtain third raw imagedata generated by at least binning the image data obtained from thespecified number of image pixels corresponding to the same channel amongthe some of the plurality of image pixels corresponding to the portionof the image sensor, and display third image generated based on thethird raw image data on the display.

The processor may identify a signal to noise ratio of the first imageand, when the signal to noise ratio of the first image is within aspecified ratio range, obtain the second raw image data using the imagesensor.

When the signal to noise ratio is not within the specified ratio range,the processor may use the image sensor to obtain the third raw imagedata generated by binning the data obtained from the specified number ofimage pixels corresponding to the same channel among the some of theplurality of image pixels corresponding to the portion of the imagesensor, and display the third image generated based on the third rawimage data on the display.

According to one embodiment, a portable communication device (e.g., theportable communication device 100 of FIG. 2) may include a camera module(the camera module 110 of FIG. 2) including an image sensor (e.g., theimage sensor 113 of FIG. 2) including a plurality of image pixels and acontrol circuit (e.g., the control circuit 115 of FIG. 2) forcontrolling the image sensor; a display (e.g., the display 140 of FIG.2); and a processor (e.g., the processor 150 of FIG. 2). The processormay receive an input associated with adjustment of zoom settingassociated with capturing, use the image sensor to obtain first rawimage data generated by binning image data obtained from the first setnumber of pixels corresponding to the same channel with respect to theplurality of image pixels based on the zoom setting being within aspecified first magnification range, display a first image generatedusing the obtained first raw image data on the display, use the imagesensor to obtain second raw image data generated by re-mosaicing imagedata obtained from some of the plurality of image pixels correspondingto a portion of the image sensor based on the zoom setting being withina specified second magnification range, and display a second imagegenerated using the obtained second raw image data on the display.

Based on the zoom setting being within a specified third magnificationrange, the processor may use the image sensor to obtain third raw imagedata in which image data obtained from pixels of the second set number,which is different from the first set number, corresponding to the samechannel is at least binned with respect to the plurality of imagepixels, and display a third image generated using the obtained third rawimage data on the display.

The processor may use the image sensor to obtain the third raw imagedata, which is generated by binning the image data obtained from thesecond set number of pixels corresponding to the same channel withrespect to the plurality of image pixels and then further re-mosaicingthe binned image data.

Based on the zoom setting being within the specified third magnificationrange, the processor may use the image sensor to obtain the third rawimage data in which the image data obtained from the second set numberof pixels corresponding to the same channel among some of the remainingimage pixels corresponding to a remaining portion of the image sensoramong the plurality of image pixels is at least binned.

The processor may identify information associated with a position and amagnification of a zoom region based on the input associated with theadjustment of the zoom setting, and the portion of the image sensor maybe a portion corresponding to the position and the magnification of thezoom region.

The processor may identify ambient illuminance, when the ambientilluminance is within a specified illuminance range, use the imagesensor to obtain the second raw image data having the specified channelpattern, which is generated by re-mosaicing the image data obtained fromthe some of the plurality of image pixels corresponding to the portionof the image sensor, when the ambient illuminance is not within thespecified illuminance range, use the image sensor to obtain the thirdraw image data generated by binning the image data obtained from thespecified number of image pixels corresponding to the same channel amongthe some of the plurality of image pixels corresponding to the portionof the image sensor, and display the third image generated based on thethird raw image data on the display.

The processor may identify a signal to noise ratio of the first image,when the signal to noise ratio is within a specified ratio range, usethe image sensor to obtain the second raw image data having thespecified channel pattern, which is generated by re-mosaicing the imagedata obtained from the some of the plurality of image pixelscorresponding to the portion of the image sensor, when the signal tonoise ratio is not within the specified ratio range, use the imagesensor to obtain the third raw image data generated by binning the dataobtained from the specified number of image pixels corresponding to thesame channel among some of the plurality of image pixels correspondingto the portion of the image sensor, and display the third imagegenerated based on the third raw image data on the display. FIG. 3Aillustrates a configuration diagram of a camera module according to anembodiment.

Referring to FIG. 3A, the camera module 110 may include the image sensor113 and the control circuit 115. In one embodiment, the camera module110 may omit some components or further include additional components.In one embodiment, some of components of the camera module 110 may becombined with each other as a single entity, but the single entity mayperform the same functions of the corresponding components before thecombination.

According to one embodiment, the image sensor 113 may include aplurality of image pixels 113 a and a read-out circuit 113 b. A chargedcoupled device (CCD) sensor or a complementary metal oxide semiconductor(CMOS) sensor may be included. A plurality of image pixels arranged inan M×M matrix may be formed in a single channel on the N×N pixels basis.To this end, a plurality ((M×M)/(N×N)) of color filters are arranged onthe plurality of image pixels, and a single color filter may be disposedon the N×N pixels. For example, 16 (=64/4) color filters (R colorfilter, G color filter, or B color filter) are arranged on a pluralityof image pixels arranged in an 8×8 matrix. Each of the 16 (=64/4) colorfilters may be disposed to have a specified channel pattern on 2×2pixels. The N×N pixels included in the same channel may be formed in amulti-cell structure capable of binning image data for at least somepixels. For example, the N×N pixels corresponding to the same channelmay share an output node (e.g., a floating diffusion area).

According to one embodiment, the control circuit 115 includes a timingcontroller 115 a, a row selector 115 b, a column selector 115 c, ananalog digital convertor (ADC) 115 d, an RGB converter 115 e, and anoutput buffer 115 f. The timing controller 115 a may generate a controlsignal for controlling an operation of at least one of the row selector115 b, the column selector 115 c, the ADC 115 d, the RGB converter 115e, and the output buffer 115 f. The row selector 115 b may selectivelyactivate one of row lines of the image sensor 113 based on the controlsignal of the timing controller 115 a. The column selector 115 c mayselectively activate one of column lines of the image sensor 113 basedon the control signal of the timing controller 115 a. The ADC 115 d mayconvert analog image data output from the image sensor 113 into digitalimage data. The RGB converter 115 e may re-mosaic digital image data tocorrespond to a specified channel pattern, thereby generating the rawimage data having the specified channel pattern. The specified channelpattern may be the bayer channel pattern in which the R image data andthe G image data alternately appear on the odd row lines, and the Gimage data and the B image data alternately appear on the even rowlines. The output buffer 115 f may buffer the raw image data, forexample, on a frame basis in the re-mosaic process. The control circuit115 to be described in a following document represents the timingcontroller 115 a and each component controlled by the timing controller115 a.

According to one embodiment, when receiving a request associated withthe activation of the image capturing function from an externalprocessor (e.g., the processor 150 of FIG. 2), the control circuit 115may activate the image capturing function of the image sensor 113. Forexample, the timing controller 115 a may supply the driving power to theimage sensor 113 to activate the image capturing function of the imagesensor 113.

According to one embodiment, when the image capturing function of theimage sensor 113 is activated, the control circuit 115 may use the imagesensor 113 to generate first raw image data having a specified channelpattern by binning image data obtained from the first set number ofimage pixels corresponding to the same channel with respect to theplurality of image pixels. The first set number may be, for example, thetotal number of image pixels included in the same channel. For example,the timing controller 115 a may control the row selector 115 b and thecolumn selector 115 c to transmit the image data for the first setnumber of image pixels individually (or simultaneously) to a floatingdiffusion area, and use the read-out circuit 113 b to read-out thefloating diffusion area after all of the image data (charges) for thefirst set number of image pixels are transmitted to the floatingdiffusion area, thereby binning (e.g., summing up) the image data forthe first set number of image pixels. The timing controller 115 a maydigitally convert the binned analog image data using the ADC 115 d andbuffer the digital data on a frame basis using the output buffer 115 fto generate the first raw image data.

According to one embodiment, the control circuit 115 may receive arequest associated with zoom setting from the processor 150 while theimage capturing function is activated. When the request associated withthe zoom setting is received, the control circuit 115 may identify someof the plurality of image pixels to read-out the image data based on therequest. The control circuit 115 may obtain image data for theidentified some of the image pixels, and re-mosaic the obtained imagedata to generate second raw image data having a specified channelpattern. The second raw image data may be image data having the sameresolution as the first raw image data or image data having a resolutionhigher than that of the first raw image data. For example, the timingcontroller 115 a may use the row selector 115 b and the column selector115 c to read-out each of the analog image data for the some of theimage pixels of the image sensor 113, and use the ADC 115 d to digitallyconvert each of the read-out analog image data. The timing controller115 a may use the RGB converter 115 e to re-mosaic the (digital) imagedata for the some of the image pixels based on the specified channelpattern, and buffer the re-mosaiced image data on a frame basis usingthe output buffer 115 f, thereby generating the second raw image datahaving the specified channel pattern.

According to one embodiment, the control circuit 115 may transmit thefirst raw image data or the second raw image data obtained using theimage sensor 113 to the processor 150. For example, the timingcontroller 115 a may transmit the first raw image data or the second rawimage data buffered on a frame basis by the output buffer 115 f to theprocessor 150.

According to various embodiments, the control circuit 115 may generatethird raw image data by binning an image obtained from the second setnumber of image pixels corresponding to the same channel among the someof the image pixels based on zoom magnification and then re-mosaicingthe binned image. For example, the control circuit 115 may generate thethird raw image data having the specified channel pattern by binning theimage data obtained from the second set number of image pixelscorresponding to the same channel with respect to some of the remainingimage pixels among the plurality of image pixels based on zoommagnification being within a third magnification range, and re-mosaicthe binned image data. The second set number may be smaller than thetotal number of image pixels included in the same channel. As anotherexample, the timing controller 115 a may use the image sensor 113 toindividually (or simultaneously) transmit, to a floating diffusion area,the image data for the second set number of image pixels included in thesame channel among the some of the remaining image pixels, and use theread-out circuit 113 b to read-out the floating diffusion area after theimage data (charges) for the second set number of image pixels aretransmitted to the floating diffusion area, thereby binning (e.g.,summing up) the image data for the second set number of image pixels.The timing controller 115 a may use the ADC 115 d to digitally converteach of the read-out analog image data. The timing controller 115 a mayuse the RGB converter 115 e to re-mosaic the (digital) image data forthe some of the remaining image pixels based on a specified channelpattern, and use the output buffer 115 f to buffer the re-mosaiced imagedata on a frame basis, thereby generating the third raw image datahaving the specified channel pattern.

According to various embodiments, each channel may include the N×K (K isa natural number different from N) number of image pixels.

FIG. 3B illustrates image pixels included in a single channel accordingto an embodiment.

Referring to FIG. 3B, each of image pixels PD1, PD2, PD3, and PD4 113 amay be a photodiode capable of accumulating light received after beingreflected on an external object. A first end of each of the image pixelsPD1, PD2, PD3, and PD4 may be connected to the ground. Further, secondends of the image pixels PD1, PD2, PD3, and PD4 may be electricallyconnected to transfer transistors TR1, TR2, TR3, and TR4, respectively.

First ends of the transfer transistors (e.g., TR1) may be connected tothe image pixels (e.g., PD1), respectively. Further, second ends of thetransfer transistors (e.g., TR1) may be connected to a common node(output node) CN. The image pixels PD1, PD2, PD3, and PD4 included inthe same channel may pass through the common node CN and share afloating diffusion area FD. The floating diffusion area FD may beelectrically connected to the common node CN to accumulate electriccharges output to the common node CN therein. The transfer transistorsTR1, TR2, TR3, and TR4 may sequentially and respectively output electriccharges stored in the image pixels to the common node CN.

For example, when the zoom magnification is within the firstmagnification range, the read-out circuit (e.g., the read-out circuit113 b of FIG. 3A) may transfer the electric charges (binned image data)accumulated in the floating diffusion area FD to the control circuit 115after all electric charges accumulated in all image pixels (of the firstset number) included in the same channel are transferred to the commonnode CN. Alternatively, for example, when the zoom magnification iswithin the second magnification range, the read-out circuit 113 b maytransfer the electric charges accumulated in the floating diffusion areaFD to the control circuit 115 after an electric charge accumulated inone image pixel is transferred to the common node CN and before anelectric charge accumulated in another image pixel is transferred to thecommon node CN. Further, for example, when the zoom magnification iswithin the third magnification range, the read-out circuit 113 b maytransfer the electric charges accumulated in the floating diffusion areaFD to the control circuit 115 when electric charges accumulated in thesecond set number of image pixels among the image pixels included ineach channel are transferred to the common node CN.

According to one embodiment, a camera module (the camera module 110 ofFIG. 3A) includes an image sensor (e.g., the image sensor 113 of FIG.3A) including a plurality of image pixels (e.g., the plurality of imagepixels 113 a of FIG. 3A) and a control circuit (e.g., the controlcircuit 115 of FIG. 3A) for controlling the image sensor. The controlcircuit may activate an image capturing function of the image sensor,when the image capturing function is activated, use the image sensor togenerate first raw image data having a specified channel pattern, whichis generated by binning image data obtained from the specified number ofimage pixels corresponding to the same channel with respect to theplurality of image pixels, transmit the generated first raw image datato an external processor (e.g., the processor 150 of FIG. 2)electrically connected to the control circuit, receive a requestassociated with zoom setting while the image capturing function isactivated, use the image sensor to generate second raw image data havingthe specified channel pattern by re-mosaicing image data for some of theplurality of image pixels corresponding to a portion of the image sensorbased at least on the request, and transmit the generated second rawimage data to the external processor.

The control circuit may generate the first raw image data or the secondraw image data based on a specified resolution.

The control circuit may identify position information of the some of theimage pixels based on the request associated with the zoom setting andobtain image data from the some of the image pixels corresponding to theidentified position information.

The control circuit may identify information associated with zoommagnification based on the request associated with the zoom setting,generate second raw image data having the specified channel pattern byre-mosaicing image data for some of the plurality of image pixelscorresponding to a portion of the image sensor based on the zoommagnification being within a first magnification range, transmit thesecond raw image data to the external processor, generate third rawimage data having the specified channel pattern by binning image dataobtained from image pixels of the different specified numbercorresponding to the same channel with respect to some of the remainingimage pixels based on the zoom magnification being within a secondmagnification range, and then re-mosaicing the image data, and transmitthe third raw image data to the external processor.

The plurality of image pixels may include N×N image pixels included ineach channel. N is a natural number equal to or greater than 2. Further,the specified number may be the total number of image pixels included ineach channel, and the different specified number may be the number ofsome of the total image pixels included in each channel.

FIG. 4 illustrates a process of generating first raw image dataaccording to an embodiment.

Referring to FIG. 4, an image sensor (e.g., the image sensor 113 of FIG.3A) may include a plurality of image pixels (e.g., 113 a of FIG. 3A)arranged in an 8×8 matrix. The plurality of image pixels 113 a may beincluded in a single channel (e.g., 411) on the 2×2 pixels basis. Forexample, one color filter (R color filter, G color filter, or B colorfilter) may be disposed on the 2×2 pixels.

According to one embodiment, the control circuit 115 may generate firstraw image data 420 having a specified channel pattern by binning imagepixels included in the same channel with respect to the plurality ofimage pixels based on a command (e.g., a request associated withactivation of the image sensor) of the processor 150. For example, thecontrol circuit 115 may bin image data obtained from four image pixelscorresponding to a R channel 411 to generate one R image data 421, andbin image data obtained from 4 image pixels corresponding to a G channel412 to generate one G image data 422. The control circuit 115 may binimage data obtained from four image pixels corresponding to a R channel413 to generate one R image data 423, and bin image data obtained from 4image pixels corresponding to a G channel 414 to generate one G imagedata 424. The control circuit 115 may bin image data obtained from fourimage pixels corresponding to a G channel 415 to generate one G imagedata 425, and bin image data obtained from 4 image pixels correspondingto a B channel 416 to generate one B image data 426. The control circuit115 may bin image data obtained from four image pixels corresponding toa G channel 417 to generate one G image data 427, and bin image dataobtained from 4 image pixels corresponding to a B channel 418 togenerate one B image data 428. Similarly, the control circuit 115 maybin image data for four image pixels corresponding to the same channelfor the remaining channels, and as a result, the image sensor 113 maygenerate the first raw image data 420 having the specified channelpattern. The specified channel pattern may be a bayer channel pattern inwhich R image data and the G image data alternately appear on odd rowlines, and the G image data and the B image data alternately appear oneven row lines.

FIG. 5 illustrates a process of generating second raw image dataaccording to an embodiment.

Referring to FIG. 5, the control circuit 115 may generate second rawimage data 440 having the specified channel pattern by re-mosaicingimage data 430 for some of the plurality of image pixels correspondingto a portion of the image sensor 113 based on a command (e.g., a requestassociated with zoom setting) of the processor 150 while the imagecapturing function is activated. For example, the control circuit 115may identify position information of the some of the image pixels basedon the request associated with the zoom setting, and obtain the imagedata 430 from the some of the image pixels based on the identifiedposition information. As the control circuit 115 performs re-mosaic ofexchanging image data that do not correspond to the specified channelpattern among the image data 430 obtained from the image sensor 113 witheach other, the control circuit 115 may generate the second raw imagedata 440 having the specified channel pattern.

Referring to FIGS. 4 and 5, according to the above-described embodiment,the first raw image data 420 and the second raw image data 440 may havethe same resolution.

FIG. 6 illustrates an example of a method for controlling an imagesensor by a processor (e.g., the processor 150 of FIG. 2) according toan embodiment.

Referring to FIG. 6, in operation 610, the processor 150 may activatethe image capturing function of the camera module 110. For example, whenthe input associated with the activation of the image capturing functionis received, the processor 150 may activate the image capturing functionof the camera module 110. The input associated with the activation ofthe image capturing function may be, for example, the user's input ofselecting (e.g., touching) the icon associated with the activation ofthe image capturing function.

In operation 620, when the image capturing function is activated, theprocessor 150 may use the image sensor 113 to obtain the first raw imagedata having the specified channel pattern, which is generated by binningthe image data obtained from the specified number (the first set number)of image pixels corresponding to the same channel with respect to theplurality of image pixels. The specified number may be, for example, thetotal number of image pixels included in each channel. The specifiedchannel pattern may be the bayer channel pattern.

In operation 630, the processor 150 may display the first imagegenerated based on the first raw image data on the display 140. Forexample, the processor 150 may color-interpolate the first raw imagedata such that each pixel of the first raw image data includes the Rinformation, the G information, and the B information, and generate thefirst image by converting the color-interpolated first raw image datainto the specified format (e.g., YUV format).

In operation 640, the processor 150 may receive the input associatedwith the adjustment of the zoom setting while the image capturingfunction is activated. For example, the input associated with theadjustment of the zoom setting may include the first input of wideningthe space between the two fingers, which touched the touch screendisplay 140.

In operation 650, the processor 150 may use the image sensor 113 toobtain the second raw image data having the specified channel pattern,which is generated by re-mosaicing the image data obtained from the someof the plurality of image pixels corresponding to the portion of theimage sensor 113 based at least on the input. For example, the processor150 may identify the selected position and the zoom magnification basedon the input associated with the adjustment of the zoom setting, andidentify the position information of the some of the image pixelscorresponding to the zoom magnification mapped on the selected position.The processor 150 may use the image sensor 113 to obtain the image dataof the some of the image pixels of the identified position information,and use the control circuit 115 to re-mosaic the obtained image data tocorrespond to the specified channel pattern, thereby obtaining thesecond raw image data.

In operation 660, the processor 150 may display the second imagegenerated based on the second raw image data on the display 140. Forexample, the processor 150 may color-interpolate the second raw imagedata such that each pixel of the second raw image data includes the Rinformation, the G information, and the B information, and generate thesecond image by converting the color-interpolated second raw image datainto the specified format (e.g., YUV format).

FIG. 7 illustrates another example of a method for controlling an imagesensor by the processor 150 according to an embodiment.

In operation 710, the processor 150 may receive the input associatedwith the adjustment of the zoom setting associating with the capturing.For example, the processor 150 may receive the input associated with theadjustment of the zoom setting while the image capturing function of thecamera module 110 is activated. The input associated with the adjustmentof the zoom setting may be, for example, the input of requesting theadjustment of the zoom magnification (e.g., zoom in or zoom out).

In operation 720, the processor 150 may use the image sensor 113 toobtain the first raw image data generated by binning the image dataobtained from the first set number of pixels corresponding to the samechannel with respect to the plurality of image pixels based on the zoomsetting being within the specified first magnification range. Theprocessor 150 may display the first image generated using the obtainedfirst raw image data on the display 140. The first set number may be thetotal number of image pixels corresponding to the same channel. Thefirst magnification range may be about 1 time, for example.

In operation 730, the processor 150 may use the image sensor 113 toobtain the second raw image data generated by re-mosaicing the imagedata obtained from the some of the plurality of image pixelscorresponding to the portion of the image sensor 113 based on the zoomsetting being within the specified second magnification range. Theprocessor 150 may display the second image generated using the obtainedsecond raw image data on the display 140. For example, the secondmagnification range may be, for example, a multiple corresponding to thenumber of horizontal image pixels N or the number of vertical imagepixels N included in each channel. For example, the processor 150 mayidentify the selected position and the zoom magnification based on theinput associated with the adjustment of the zoom setting, and identifythe position information of the some of the image pixels correspondingto the zoom magnification mapped on the selected position. The processor150 may use the image sensor 113 to obtain the image data of the some ofthe image pixels of the identified position information, and use thecontrol circuit 115 to re-mosaic the obtained image data to correspondto the specified channel pattern, thereby obtaining the second raw imagedata. The second raw image data may be image data having the sameresolution as the first raw image data.

According to various embodiments, in operation 730, when the zoomsetting is within the specified second magnification range, theprocessor 150 may use the image sensor 113 to re-mosaic the image dataobtained from the plurality of image pixels to obtain the raw imagedata, select the raw image data corresponding to the portion of theimage sensor 113 from the obtained raw image data, and use the selectedimage data to generate the second image.

FIG. 8 illustrates a method for controlling an image sensor by thecontrol circuit 115 according to an embodiment.

In operation 810, the control circuit (e.g., the control circuit 115 ofFIG. 2) may activate the image capturing function of the image sensor113. When the request associated with the activation of the imagecapturing function is received from the processor 150, the controlcircuit 115 may activate the image capturing function of the imagesensor 113.

In operation 820, when the image capturing function is activated, thecontrol circuit 115 may use the image sensor 113 to generate the firstraw image data having the specified channel pattern, which is generatedby binning the image data obtained from the specified number of imagepixels corresponding to the same channel with respect to the pluralityof image pixels. The specified number may be, for example, the totalnumber of image pixels corresponding to the same channel. The specifiedchannel pattern may be the bayer channel pattern.

In operation 830, the control circuit 115 may transmit the first rawimage data to the external processor (e.g., the processor 150 of FIG. 2)electrically connected to the control circuit.

In operation 840, the control circuit 115 may receive the requestassociated with the zoom setting while the image capturing function isactivated.

In operation 850, the control circuit 115 may use the image sensor 113to re-mosaic the image data for the some of the plurality of imagepixels corresponding to the portion of the image sensor based at leaston the request, thereby generating the second raw image data having thespecified channel pattern. The second raw image data may have the sameresolution as the first raw image data.

In operation 860, the control circuit 115 may transmit the generatedsecond raw image data to the external processor 150.

FIG. 9 illustrates a block diagram of an electronic device 901 in anetwork environment 900 according to various embodiments. Referring toFIG. 9, the electronic device 901 in the network environment 900 maycommunicate with an electronic device 902 via a first network 998 (e.g.,a short-range wireless communication network), or an electronic device904 or a server 908 via a second network 999 (e.g., a long-rangewireless communication network). According to an embodiment, theelectronic device 901 may communicate with the electronic device 904 viathe server 908. According to an embodiment, the electronic device 901(e.g., the communication device 100 of FIG. 2) may include a processor920, memory 930 (e.g., the memory 130 of FIG. 2), an input device 950, asound output device 955, a display device 960 (e.g., the display 140 ofFIG. 2), an audio module 970, a sensor module 976 (e.g., the illuminancesensor 120 of FIG. 2), an interface 977, a haptic module 979, a cameramodule 980 (e.g., the camera module 110 of FIG. 2), a power managementmodule 988, a battery 989, a communication module 990, a subscriberidentification module (SIM) 996, or an antenna module 997. In someembodiments, at least one (e.g., the display device 960 or the cameramodule 980) of the components may be omitted from the electronic device901, or one or more other components may be added in the electronicdevice 901. In some embodiments, some of the components may beimplemented as single integrated circuitry. For example, the sensormodule 976 (e.g., a fingerprint sensor, an iris sensor, or anilluminance sensor) may be implemented as embedded in the display device960 (e.g., a display).

The processor 920 may execute, for example, software (e.g., a program940) to control at least one other component (e.g., a hardware orsoftware component) of the electronic device 901 coupled with theprocessor 920, and may perform various data processing or computation.According to one embodiment, as at least part of the data processing orcomputation, the processor 920 may load a command or data received fromanother component (e.g., the sensor module 976 or the communicationmodule 990) in volatile memory 932, process the command or the datastored in the volatile memory 932, and store resulting data innon-volatile memory 934. According to an embodiment, the processor 920may include a main processor 921 (e.g., a central processing unit (CPU)or an application processor (AP)), and an auxiliary processor 923 (e.g.,a graphics processing unit (GPU), an image signal processor (ISP), asensor hub processor, or a communication processor (CP)) that isoperable independently from, or in conjunction with, the main processor921. Additionally or alternatively, the auxiliary processor 923 may beadapted to consume less power than the main processor 921, or to bespecific to a specified function. The auxiliary processor 923 may beimplemented as separate from, or as part of the main processor 921.

The auxiliary processor 923 may control at least some of functions orstates related to at least one component (e.g., the display device 960,the sensor module 976, or the communication module 990) among thecomponents of the electronic device 901, instead of the main processor921 while the main processor 921 is in an inactive (e.g., sleep) state,or together with the main processor 921 while the main processor 921 isin an active state (e.g., executing an application). According to anembodiment, the auxiliary processor 923 (e.g., an image signal processoror a communication processor) may be implemented as part of anothercomponent (e.g., the camera module 980 or the communication module 990)functionally related to the auxiliary processor 923.

The memory 930 may store various data used by at least one component(e.g., the processor 920 or the sensor module 976) of the electronicdevice 901. The various data may include, for example, software (e.g.,the program 940) and input data or output data for a command relatedthereto. The memory 930 may include the volatile memory 932 or thenon-volatile memory 934.

The program 940 may be stored in the memory 930 as software, and mayinclude, for example, an operating system (OS) 942, middleware 944, oran application 946.

The input device 950 may receive a command or data to be used by othercomponent (e.g., the processor 920) of the electronic device 901, fromthe outside (e.g., a user) of the electronic device 901. The inputdevice 950 may include, for example, a microphone, a mouse, a keyboard,or a digital pen (e.g., a stylus pen).

The sound output device 955 may output sound signals to the outside ofthe electronic device 901. The sound output device 955 may include, forexample, a speaker or a receiver. The speaker may be used for generalpurposes, such as playing multimedia or playing record, and the receivermay be used for an incoming calls. According to an embodiment, thereceiver may be implemented as separate from, or as part of the speaker.

The display device 960 may visually provide information to the outside(e.g., a user) of the electronic device 901. The display device 960 mayinclude, for example, a display, a hologram device, or a projector andcontrol circuitry to control a corresponding one of the display,hologram device, and projector. According to an embodiment, the displaydevice 960 may include touch circuitry adapted to detect a touch, orsensor circuitry (e.g., a pressure sensor) adapted to measure theintensity of force incurred by the touch.

The audio module 970 may convert a sound into an electrical signal andvice versa. According to an embodiment, the audio module 970 may obtainthe sound via the input device 950, or output the sound via the soundoutput device 955 or a headphone of an external electronic device (e.g.,an electronic device 902) directly (e.g., wiredly) or wirelessly coupledwith the electronic device 901.

The sensor module 976 may detect an operational state (e.g., power ortemperature) of the electronic device 901 or an environmental state(e.g., a state of a user) external to the electronic device 901, andthen generate an electrical signal or data value corresponding to thedetected state. According to an embodiment, the sensor module 976 mayinclude, for example, a gesture sensor, a gyro sensor, an atmosphericpressure sensor, a magnetic sensor, an acceleration sensor, a gripsensor, a proximity sensor, a color sensor, an infrared (IR) sensor, abiometric sensor, a temperature sensor, a humidity sensor, or anilluminance sensor.

The interface 977 may support one or more specified protocols to be usedfor the electronic device 901 to be coupled with the external electronicdevice (e.g., the electronic device 902) directly (e.g., wiredly) orwirelessly. According to an embodiment, the interface 977 may include,for example, a high definition multimedia interface (HDMI), a universalserial bus (USB) interface, a secure digital (SD) card interface, or anaudio interface.

A connecting terminal 978 may include a connector via which theelectronic device 901 may be physically connected with the externalelectronic device (e.g., the electronic device 902). According to anembodiment, the connecting terminal 978 may include, for example, a HDMIconnector, a USB connector, a SD card connector, or an audio connector(e.g., a headphone connector).

The haptic module 979 may convert an electrical signal into a mechanicalstimulus (e.g., a vibration or a movement) or electrical stimulus whichmay be recognized by a user via his tactile sensation or kinestheticsensation. According to an embodiment, the haptic module 979 mayinclude, for example, a motor, a piezoelectric element, or an electricstimulator.

The camera module 980 may capture a still image or moving images.According to an embodiment, the camera module 980 may include one ormore lenses, image sensors, image signal processors, or flashes.

The power management module 988 may manage power supplied to theelectronic device 901. According to one embodiment, the power managementmodule 988 may be implemented as at least part of, for example, a powermanagement integrated circuit (PMIC).

The battery 989 may supply power to at least one component of theelectronic device 901. According to an embodiment, the battery 989 mayinclude, for example, a primary cell which is not rechargeable, asecondary cell which is rechargeable, or a fuel cell.

The communication module 990 may support establishing a direct (e.g.,wired) communication channel or a wireless communication channel betweenthe electronic device 901 and the external electronic device (e.g., theelectronic device 902, the electronic device 904, or the server 908) andperforming communication via the established communication channel. Thecommunication module 990 may include one or more communicationprocessors that are operable independently from the processor 920 (e.g.,the application processor (AP)) and supports a direct (e.g., wired)communication or a wireless communication. According to an embodiment,the communication module 990 may include a wireless communication module992 (e.g., a cellular communication module, a short-range wirelesscommunication module, or a global navigation satellite system (GNSS)communication module) or a wired communication module 994 (e.g., a localarea network (LAN) communication module or a power line communication(PLC) module). A corresponding one of these communication modules maycommunicate with the external electronic device via the first network998 (e.g., a short-range communication network, such as Bluetooth™,wireless-fidelity (Wi-Fi) direct, or infrared data association (IrDA))or the second network 999 (e.g., a long-range communication network,such as a cellular network, the Internet, or a computer network (e.g.,LAN or wide area network (WAN)). These various types of communicationmodules may be implemented as a single component (e.g., a single chip),or may be implemented as multi components (e.g., multi chips) separatefrom each other. The wireless communication module 992 may identify andauthenticate the electronic device 901 in a communication network, suchas the first network 998 or the second network 999, using subscriberinformation (e.g., international mobile subscriber identity (IMSI))stored in the subscriber identification module 996.

The antenna module 997 may transmit or receive a signal or power to orfrom the outside (e.g., the external electronic device) of theelectronic device 901. According to an embodiment, the antenna module997 may include an antenna including a radiating element composed of aconductive material or a conductive pattern formed in or on a substrate(e.g., PCB). According to an embodiment, the antenna module 997 mayinclude a plurality of antennas. In such a case, at least one antennaappropriate for a communication scheme used in the communicationnetwork, such as the first network 998 or the second network 999, may beselected, for example, by the communication module 990 (e.g., thewireless communication module 992) from the plurality of antennas. Thesignal or the power may then be transmitted or received between thecommunication module 990 and the external electronic device via theselected at least one antenna. According to an embodiment, anothercomponent (e.g., a radio frequency integrated circuit (RFIC)) other thanthe radiating element may be additionally formed as part of the antennamodule 997.

At least some of the above-described components may be coupled mutuallyand communicate signals (e.g., commands or data) therebetween via aninter-peripheral communication scheme (e.g., a bus, general purposeinput and output (GPIO), serial peripheral interface (SPI), or mobileindustry processor interface (MIPI)).

According to an embodiment, commands or data may be transmitted orreceived between the electronic device 901 and the external electronicdevice 904 via the server 908 coupled with the second network 999. Eachof the electronic devices 902 and 904 may be a device of a same type as,or a different type, from the electronic device 901. According to anembodiment, all or some of operations to be executed at the electronicdevice 901 may be executed at one or more of the external electronicdevices 902, 904, or 908. For example, if the electronic device 901should perform a function or a service automatically, or in response toa request from a user or another device, the electronic device 901,instead of, or in addition to, executing the function or the service,may request the one or more external electronic devices to perform atleast part of the function or the service. The one or more externalelectronic devices receiving the request may perform the at least partof the function or the service requested, or an additional function oran additional service related to the request, and transfer an outcome ofthe performing to the electronic device 901. The electronic device 901may provide the outcome, with or without further processing of theoutcome, as at least part of a reply to the request. To that end, acloud computing, distributed computing, or client-server computingtechnology may be used, for example.

FIG. 10 illustrates a block diagram 1000 of the camera module 980according to various embodiments. Referring to FIG. 10, the cameramodule 980 may include a lens assembly 1010, a flash 1020, an imagesensor 1030 (e.g., the control circuit 115 and the image sensor 113 ofFIG. 2), an image stabilizer 1040, memory 1050 (e.g., buffer memory), oran image signal processor 1060 (e.g., the processor 150 of FIG. 2). Thelens assembly 1010 may collect light emitted or reflected from an objectwhose image is to be taken. The lens assembly 1010 may include one ormore lenses. According to an embodiment, the camera module 980 mayinclude a plurality of lens assemblies 1010. In such a case, the cameramodule 980 may form, for example, a dual camera, a 360-degree camera, ora spherical camera. Some of the plurality of lens assemblies 1010 mayhave the same lens attribute (e.g., view angle, focal length,auto-focusing, f number, or optical zoom), or at least one lens assemblymay have one or more lens attributes different from those of anotherlens assembly. The lens assembly 1010 may include, for example, awide-angle lens or a telephoto lens.

The flash 1020 may emit light that is used to reinforce light reflectedfrom an object. According to an embodiment, the flash 1020 may includeone or more light emitting diodes (LEDs) (e.g., a red-green-blue (RGB)LED, a white LED, an infrared (IR) LED, or an ultraviolet (UV) LED) or axenon lamp. The image sensor 1030 may obtain an image corresponding toan object by converting light emitted or reflected from the object andtransmitted via the lens assembly 1010 into an electrical signal.According to an embodiment, the image sensor 1030 may include oneselected from image sensors having different attributes, such as a RGBsensor, a black-and-white (BW) sensor, an IR sensor, or a UV sensor, aplurality of image sensors having the same attribute, or a plurality ofimage sensors having different attributes. Each image sensor included inthe image sensor 1030 may be implemented using, for example, a chargedcoupled device (CCD) sensor or a complementary metal oxide semiconductor(CMOS) sensor.

The image stabilizer 1040 may move the image sensor 1030 or at least onelens included in the lens assembly 1010 in a particular direction, orcontrol an operational attribute (e.g., adjust the read-out timing) ofthe image sensor 1030 in response to the movement of the camera module980 or the electronic device 901 including the camera module 980. Thisallows compensating for at least part of a negative effect (e.g., imageblurring) by the movement on an image being captured. According to anembodiment, the image stabilizer 1040 may sense such a movement by thecamera module 980 or the electronic device 901 using a gyro sensor (notshown) or an acceleration sensor (not shown) disposed inside or outsidethe camera module 980. According to an embodiment, the image stabilizer1040 may be implemented, for example, as an optical image stabilizer.

The memory 1050 may store, at least temporarily, at least part of animage obtained via the image sensor 1030 for a subsequent imageprocessing task. For example, if image capturing is delayed due toshutter lag or multiple images are quickly captured, a raw imageobtained (e.g., a Bayer-patterned image, a high-resolution image) may bestored in the memory 1050, and its corresponding copy image (e.g., alow-resolution image) may be previewed via the display device 960.Thereafter, if a specified condition is met (e.g., by a user's input orsystem command), at least part of the raw image stored in the memory1050 may be obtained and processed, for example, by the image signalprocessor 1060. According to an embodiment, the memory 1050 may beconfigured as at least part of the memory 930 or as a separate memorythat is operated independently from the memory 930.

The image signal processor 1060 may perform one or more image processingwith respect to an image obtained via the image sensor 1030 or an imagestored in the memory 1050. The one or more image processing may include,for example, depth map generation, three-dimensional (3D) modeling,panorama generation, feature point extraction, image synthesizing, orimage compensation (e.g., noise reduction, resolution adjustment,brightness adjustment, blurring, sharpening, or softening). Additionallyor alternatively, the image signal processor 1060 may perform control(e.g., exposure time control or read-out timing control) with respect toat least one (e.g., the image sensor 1030) of the components included inthe camera module 980. An image processed by the image signal processor1060 may be stored back in the memory 1050 for further processing, ormay be provided to an external component (e.g., the memory 930, thedisplay device 960, the electronic device 902, the electronic device904, or the server 908) outside the camera module 980. According to anembodiment, the image signal processor 1060 may be configured as atleast part of the processor 920, or as a separate processor that isoperated independently from the processor 920. If the image signalprocessor 1060 is configured as a separate processor from the processor920, at least one image processed by the image signal processor 1060 maybe displayed, by the processor 920, via the display device 960 as it isor after being further processed.

According to an embodiment, the electronic device 901 may include aplurality of camera modules 980 having different attributes orfunctions. In such a case, at least one of the plurality of cameramodules 980 may form, for example, a wide-angle camera and at leastanother of the plurality of camera modules 980 may form a telephotocamera. Similarly, at least one of the plurality of camera modules 980may form, for example, a front camera and at least another of theplurality of camera modules 980 may form a rear camera.

The electronic device according to various embodiments may be one ofvarious types of electronic devices. The electronic devices may include,for example, a portable communication device (e.g., a smartphone), acomputer device, a portable multimedia device, a portable medicaldevice, a camera, a wearable device, or a home appliance. According toan embodiment of the disclosure, the electronic devices are not limitedto those described above.

It should be appreciated that various embodiments of the presentdisclosure and the terms used therein are not intended to limit thetechnological features set forth herein to particular embodiments andinclude various changes, equivalents, or replacements for acorresponding embodiment. With regard to the description of thedrawings, similar reference numerals may be used to refer to similar orrelated elements. It is to be understood that a singular form of a nouncorresponding to an item may include one or more of the things, unlessthe relevant context clearly indicates otherwise. As used herein, eachof such phrases as “A or B,” “at least one of A and B,” “at least one ofA or B,” “A, B, or C,” “at least one of A, B, and C,” and “at least oneof A, B, or C,” may include any one of, or all possible combinations ofthe items enumerated together in a corresponding one of the phrases. Asused herein, such terms as “1st” and “2nd,” or “first” and “second” maybe used to simply distinguish a corresponding component from another,and does not limit the components in other aspect (e.g., importance ororder). It is to be understood that if an element (e.g., a firstelement) is referred to, with or without the term “operatively” or“communicatively”, as “coupled with,” “coupled to,” “connected with,” or“connected to” another element (e.g., a second element), it means thatthe element may be coupled with the other element directly (e.g.,wiredly), wirelessly, or via a third element.

As used herein, the term “module” may include a unit implemented inhardware, software, or firmware, and may interchangeably be used withother terms, for example, “logic,” “logic block,” “part,” or“circuitry”. A module may be a single integral component, or a minimumunit or part thereof, adapted to perform one or more functions. Forexample, according to an embodiment, the module may be implemented in aform of an application-specific integrated circuit (ASIC).

Various embodiments as set forth herein may be implemented as software(e.g., the program 940) including one or more instructions that arestored in a storage medium (e.g., internal memory 936 or external memory938) that is readable by a machine (e.g., the electronic device 901).For example, a processor(e.g., the processor 920) of the machine (e.g.,the electronic device 901) may invoke at least one of the one or moreinstructions stored in the storage medium, and execute it, with orwithout using one or more other components under the control of theprocessor. This allows the machine to be operated to perform at leastone function according to the at least one instruction invoked. The oneor more instructions may include a code generated by a compiler or acode executable by an interpreter. The machine-readable storage mediummay be provided in the form of a non-transitory storage medium. Wherein,the term “non-transitory” simply means that the storage medium is atangible device, and does not include a signal (e.g., an electromagneticwave), but this term does not differentiate between where data issemi-permanently stored in the storage medium and where the data istemporarily stored in the storage medium.

According to an embodiment, a method according to various embodiments ofthe disclosure may be included and provided in a computer programproduct. The computer program product may be traded as a product betweena seller and a buyer. The computer program product may be distributed inthe form of a machine-readable storage medium (e.g., compact disc readonly memory (CD-ROM)), or be distributed (e.g., downloaded or uploaded)online via an application store (e.g., PlayStore™), or between two userdevices (e.g., smart phones) directly. If distributed online, at leastpart of the computer program product may be temporarily generated or atleast temporarily stored in the machine-readable storage medium, such asmemory of the manufacturer's server, a server of the application store,or a relay server.

According to various embodiments, each component (e.g., a module or aprogram) of the above-described components may include a single entityor multiple entities. According to various embodiments, one or more ofthe above-described components may be omitted, or one or more othercomponents may be added. Alternatively or additionally, a plurality ofcomponents (e.g., modules or programs) may be integrated into a singlecomponent. In such a case, according to various embodiments, theintegrated component may still perform one or more functions of each ofthe plurality of components in the same or similar manner as they areperformed by a corresponding one of the plurality of components beforethe integration. According to various embodiments, operations performedby the module, the program, or another component may be carried outsequentially, in parallel, repeatedly, or heuristically, or one or moreof the operations may be executed in a different order or omitted, orone or more other operations may be added.

According to embodiments disclosed in the disclosure, the resolutiondegradation of the image data due to the zoom setting may be lowered. Inaddition, various effects that are directly or indirectly graspedthrough the disclosure may be provided.

Although the present disclosure has been described with variousembodiments, various changes and modifications may be suggested to oneskilled in the art. It is intended that the present disclosure encompasssuch changes and modifications as fall within the scope of the appendedclaims.

What is claimed is:
 1. A portable communication device comprising: acamera module including an image sensor including a plurality of imagepixels and a control circuit configured to control the image sensor; adisplay; and a processor configured to: activate an image capturingfunction of the camera module; based on the image capturing functionbeing activated, obtain first raw image data comprising a specifiedchannel pattern generated by binning image data obtained from aspecified number of image pixels corresponding to a same channel withrespect to the plurality of image pixels, by using the image sensor;display a first image generated based on the first raw image data on thedisplay; while the image capturing function is activated, receive aninput associated with an adjustment of a zoom setting; obtain second rawimage data comprising a specified channel pattern, generated byre-mosaicing image data obtained from some of the plurality of imagepixels corresponding to a portion of the image sensor based at least onthe input, by using the image sensor; and display a second imagegenerated based on the second raw image data on the display.
 2. Theportable communication device of claim 1, wherein the processor isconfigured to identify a position selected as a zoom region and a zoommagnification based at least on the input associated with the adjustmentof the zoom setting, wherein the some of the plurality of image pixelsare image pixels corresponding to the identified zoom magnificationmapped on the selected position.
 3. The portable communication device ofclaim 1, wherein the second raw image data is raw image data generatedto have a resolution corresponding to a resolution of the first rawimage data.
 4. The portable communication device of claim 1, wherein theprocessor is configured to: identify ambient illuminance; and when theambient illuminance is within a specified illuminance range, use theimage sensor to obtain the second raw image data.
 5. The portablecommunication device of claim 4, wherein the processor is configured to:when the ambient illuminance is not within the specified illuminancerange, use the image sensor to obtain third raw image data generated byat least binning image data obtained from the specified number of imagepixels corresponding to the same channel among the some of the pluralityof image pixels corresponding to the portion of the image sensor; anddisplay a third image generated based on the third raw image data on thedisplay.
 6. The portable communication device of claim 1, wherein theprocessor is configured to: identify a signal to noise ratio of thefirst image; and when the signal to noise ratio of the first image iswithin a specified ratio range, use the image sensor to obtain thesecond raw image data.
 7. The portable communication device of claim 6,wherein the processor is configured to: when the signal to noise ratiois not within the specified ratio range, use the image sensor to obtainthird raw image data generated by binning image data obtained from thespecified number of image pixels corresponding to the same channel amongthe some of the plurality of image pixels corresponding to the portionof the image sensor; and display a third image generated based on thethird raw image data on the display.
 8. A portable communication devicecomprising: a camera module including an image sensor including aplurality of image pixels and a control circuit configured to controlthe image sensor; a display; and a processor configured to: receive aninput associated with an adjustment of a zoom setting related tocapturing an image; obtain first raw image data generated by binningimage data obtained from a first set number of pixels corresponding to asame channel with respect to the plurality of image pixels based on thezoom setting being within a specified first magnification range by usingthe image sensor, and display a first image generated based on theobtained first raw image data on the display; and obtain second rawimage data generated by re-mosaicing image data obtained from some ofthe plurality of image pixels corresponding to a portion of the imagesensor based on the zoom setting being within a specified secondmagnification range by using the image sensor, and display a secondimage generated based on the obtained second raw image data on thedisplay.
 9. The portable communication device of claim 8, wherein theprocessor is configured to: use the image sensor to obtain third rawimage data generated by at least binning image data obtained from asecond set number of pixels corresponding to the same channel withrespect to the plurality of image pixels based on the zoom setting beingwithin a specified third magnification range, wherein the second setnumber of pixels is different from the first set number of pixels; anddisplay a third image generated using the third raw image data on thedisplay.
 10. The portable communication device of claim 9, wherein theprocessor is configured to: use the image sensor to obtain the third rawimage data, generated by binning the image data obtained from the secondset number of pixels corresponding to the same channel with respect tothe plurality of image pixels and then further re-mosaicing the binnedimage data.
 11. The portable communication device of claim 9, whereinthe processor is configured to: use the image sensor to obtain the thirdraw image data generated by at least binning image data obtained fromthe second set number of pixels corresponding to the same channel amongsome of a remaining image pixels corresponding to another portion of theimage sensor among the plurality of image pixels based on the zoomsetting being within the specified third magnification range.
 12. Theportable communication device of claim 8, wherein the processor isconfigured to: identify information associated with a position and amagnification of a zoom region based at least on the input associatedwith the adjustment of the zoom setting, and wherein the portion of theimage sensor is a portion corresponding to the position and themagnification of the zoom region.
 13. The portable communication deviceof claim 8, wherein the processor is configured to: identify ambientilluminance; based on the ambient illuminance being within a specifiedilluminance range, obtain the second raw image data comprising aspecified channel pattern, generated by re-mosaicing the image dataobtained from the some of the plurality of image pixels corresponding tothe portion of the image sensor, by using the image sensor; based on theambient illuminance not being within the specified illuminance range,obtain third raw image data generated by binning image data obtainedfrom a specified number of image pixels corresponding to the samechannel among the some of the plurality of image pixels corresponding tothe portion of the image sensor, by using the image sensor; and displaya third image generated based on the third raw image data on thedisplay.
 14. The portable communication device of claim 8, wherein theprocessor is configured to: identify a signal to noise ratio of thefirst image; based on the signal to noise ratio is within a specifiedratio range, obtain the second raw image data comprising a specifiedchannel pattern, generated by re-mosaicing the image data obtained fromthe some of the plurality of image pixels corresponding to the portionof the image sensor, by using the image sensor; based on the signal tonoise ratio not being within the specified ratio range, obtain third rawimage data, generated by binning data obtained from a specified numberof image pixels corresponding to the same channel among the some of theplurality of image pixels corresponding to the portion of the imagesensor, by using the image sensor; and display a third image generatedbased on the third raw image data on the display.
 15. A camera modulecomprising: an image sensor including a plurality of image pixels; and acontrol circuit for controlling the image sensor, wherein the controlcircuit is configured to: activate an image capturing function of theimage sensor; based on the image capturing function being activated,generate first raw image data having a specified channel patterngenerated by binning image data obtained from a specified number ofimage pixels corresponding to a same channel with respect to theplurality of image pixels, by using the image sensor; transmit thegenerated first raw image data to an external processor electricallyconnected to the control circuit; receive a request associated with azoom setting while the image capturing function is activated; generatesecond raw image data having the specified channel pattern byre-mosaicing image data for some of the plurality of image pixelscorresponding to a portion of the image sensor based at least on therequest, by using the image sensor; and transmit the generated secondraw image data to the external processor.
 16. The camera module of claim15, wherein the control circuit is configured to generate the first rawimage data or the second raw image data based on a specified resolution.17. The camera module of claim 15, wherein the control circuit isconfigured to: identify position information of the some of theplurality of image pixels based on the request associated with the zoomsetting; and obtain image data from the some of the plurality of imagepixels corresponding to the identified position information.
 18. Thecamera module of claim 15, wherein the control circuit is configured to:identify information associated with a zoom magnification based on therequest associated with the zoom setting; generate the second raw imagedata having the specified channel pattern by re-mosaicing the image datafor the some of the plurality of image pixels corresponding to theportion of the image sensor based on the zoom magnification being withina first magnification range; transmit the second raw image data to theexternal processor; generate third raw image data having the specifiedchannel pattern by binning image data obtained from a differentspecified number of image pixels corresponding to a same channel withrespect to some of a remaining image pixels based on the zoommagnification being within a second magnification range, and thenre-mosaicing the binned image data; and transmit the third raw imagedata to the external processor.
 19. The camera module of claim 15,wherein the plurality of image pixels includes N×N image pixels includedin each channel, wherein N is a natural number equal to or greater than2, wherein the specified number of image pixels is a total number ofimage pixels included in each channel, and wherein a different specifiednumber of image pixels is a number of some of the total number of imagepixels included in each channel.